Stark: Fast and Scalable Strassen’s Matrix Multiplication Using Apache Spark
نویسندگان
چکیده
This article presents a new fast, highly scalable distributed matrix multiplication algorithm on Apache Spark, called Stark , based Strassen’s algorithm. Stark preserves seven multiplications scheme in environment and thus achieves asymptotically faster execution time. It creates recursion tree of computation where each level the corresponds to division combination blocks stored form Resilient Distributed Datasets (RDDs). processes divide combine step parallel memorises sub-matrices by intelligently tagging it. To best our knowledge, is first implementation distribute Spark platform. We also report detailed complexity analysis for proposed algorithm, taking into account communication costs. Experimental results suggest that outperforms existing implementations – Marlin MLLib high sizes ( $\geq 16384\times 16384$ ). Our experiments reveal optimal block size, which shown from theoretical analysis. show experimental running times match closely. has been experimentally exhibits strong scalability with increasing number executors.
منابع مشابه
Scalable SDE Filtering and Inference with Apache Spark
In this paper, we consider the problem of Bayesian filtering and inference for time series data modeled as noisy, discrete-time observations of a stochastic differential equation (SDE) with undetermined parameters. We develop a Metropolis algorithm to sample from the high-dimensional joint posterior density of all SDE parameters and state time series. Our approach relies on an innovative densit...
متن کاملFast matrix multiplication using coherent configurations
We introduce a relaxation of the notion of tensor rank, called s-rank, and show that upper bounds on the s-rank of the matrix multiplication tensor imply upper bounds on the ordinary rank. In particular, if the “s-rank exponent of matrix multiplication” equals 2, then ω = 2. This connection between the s-rank exponent and the ordinary exponent enables us to significantly generalize the group-th...
متن کاملReal-time News Recommendations using Apache Spark
Recommending news articles is a challenging task due to the continuous changes in the set of available news articles and the contextdependent preferences of users. Traditional recommender approaches are optimized for analyzing static data sets. In news recommendation scenarios, characterized by continuous changes, high volume of messages, and tight time constraints, alternative approaches are n...
متن کاملFast Matrix Multiplication
We give an overview of the history of fast algorithms for matrix multiplication. Along the way, we look at some other fundamental problems in algebraic complexity like polynomial evaluation. This exposition is self-contained. To make it accessible to a broad audience, we only assume a minimal mathematical background: basic linear algebra, familiarity with polynomials in several variables over r...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Big Data
سال: 2022
ISSN: ['2372-2096', '2332-7790']
DOI: https://doi.org/10.1109/tbdata.2020.2977326